Sequential Tests of Multiple Hypotheses Controlling Type I and II Familywise Error Rates.
نویسندگان
چکیده
This paper addresses the following general scenario: A scientist wishes to perform a battery of experiments, each generating a sequential stream of data, to investigate some phenomenon. The scientist would like to control the overall error rate in order to draw statistically-valid conclusions from each experiment, while being as efficient as possible. The between-stream data may differ in distribution and dimension but also may be highly correlated, even duplicated exactly in some cases. Treating each experiment as a hypothesis test and adopting the familywise error rate (FWER) metric, we give a procedure that sequentially tests each hypothesis while controlling both the type I and II FWERs regardless of the between-stream correlation, and only requires arbitrary sequential test statistics that control the error rates for a given stream in isolation. The proposed procedure, which we call the sequential Holm procedure because of its inspiration from Holm's (1979) seminal fixed-sample procedure, shows simultaneous savings in expected sample size and less conservative error control relative to fixed sample, sequential Bonferroni, and other recently proposed sequential procedures in a simulation study.
منابع مشابه
A Rejection Principle for Sequential Tests of Multiple Hypotheses Controlling Familywise Error Rates.
We present a unifying approach to multiple testing procedures for sequential (or streaming) data by giving sufficient conditions for a sequential multiple testing procedure to control the familywise error rate (FWER). Together we call these conditions a "rejection principle for sequential tests," which we then apply to some existing sequential multiple testing procedures to give simplified unde...
متن کاملFDR_TEST: A SAS Macro for Calculating New Methods of Error Control in Multiple Hypothesis Testing
The testing of multiple null hypotheses in a single study is a common occurrence in applied research. The problem of Type I error inflation or probability pyramiding in such contexts has been well-known for many years. General procedures for the control of Type I error rates in multiple testing are the Bonferroni procedure and its’ more recent modifications. These procedures partition a desired...
متن کاملMany tests of significance: new methods for controlling type I errors.
There have been many discussions of how Type I errors should be controlled when many hypotheses are tested (e.g., all possible comparisons of means, correlations, proportions, the coefficients in hierarchical models, etc.). By and large, researchers have adopted familywise (FWER) control, though this practice certainly is not universal. Familywise control is intended to deal with the multiplici...
متن کاملPer Family or Familywise Type I Error Control: "Eether, Eyether, Neether, Nyther, Let's Call the Whole Thing Off!"
Frane (2015) pointed out the difference between per-family and familywise Type I error control and how different multiple comparison procedures control one method but not necessarily the other. He then went on to demonstrate in the context of a two group multivariate design containing different numbers of dependent variables and correlations between variables how the per-family rate inflates be...
متن کاملType I Error Rates and Power of Multiple Hypothesis Testing Procedures in Factorial ANOVA
There are numerous General Linear Model (GLM) statistical designs that may require multiple hypothesis testing (MHT) procedures that control the Type I error inflation that occurs with multiple tests. This study investigated familywise error rates (FWER) and statistical power rates of several alpha-adjustment MHT procedures in factorial ANOVA, but results apply broadly to GLM procedures. Of fou...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید
ثبت ناماگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید
ورودعنوان ژورنال:
- Journal of statistical planning and inference
دوره 153 شماره
صفحات -
تاریخ انتشار 2014